Inference, Bayes, Expectations, and Randomness

In [1]:
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline
%load_ext watermark
In [2]:
%watermark --author "Ryan Sloot |" -d -v

Ryan Sloot | 2016-10-08 

CPython 3.5.1
IPython 4.2.0

Independent Random Variables

  • R.V.s: $W, I, X, Y$
  • joint pdfs: $ p_{W,I}$ and $p_{X,Y} $
In [3]:
## W-->sunny, rainy, snowy | I-->1,0
prob_W_I = np.array([[1/2, 0], [0, 1/6], [0,1/3]])
In [4]:
## Marginal distributions
## of p_W and p_I
prob_W = prob_W_I.sum(axis=1)
prob_I = prob_W_I.sum(axis=0)
In [5]:
prob_I
Out[5]:
array([ 0.5,  0.5])
In [6]:
np.outer(prob_W, prob_I)==prob_W_I
## not independent
Out[6]:
array([[False, False],
       [False, False],
       [False, False]], dtype=bool)
In [7]:
prob_X_Y = np.array([[1/4,1/4], [1/12,1/12],[1/6,1/6]])
In [8]:
prob_X = prob_X_Y.sum(axis=1)
prob_Y = prob_X_Y.sum(axis=0)
In [9]:
np.outer(prob_X, prob_Y) == prob_X_Y
Out[9]:
array([[ True,  True],
       [ True,  True],
       [ True,  True]], dtype=bool)

Ice Cream Sales in Inferenceville

$\qquad S$: Sales, $C$: Crime, $T$: Temperature

In [10]:
.1/.35
Out[10]:
0.28571428571428575
In [11]:
.4/.65
Out[11]:
0.6153846153846154
In [12]:
8/13
Out[12]:
0.6153846153846154
In [13]:
P_S_C = np.array([[0.4, 0.1], [.25, .25]])
P_S = P_S_C.sum(axis=1)#rows
P_C = P_S_C.sum(axis=0)#cols
In [14]:
np.outer(P_S, P_C) == P_S_C
Out[14]:
array([[False, False],
       [False, False]], dtype=bool)
In [15]:
p_s_c_given_T0 = np.array([[0.72, 0.08], [0.18, 0.02]])
p_s_c_given_T1 = np.array([[0.08, 0.12], [0.32, 0.48]])
In [16]:
p_s_givenT0 = p_s_c_given_T0.sum(axis=1)
p_s_givenT1 = p_s_c_given_T1.sum(axis=1)
p_c_givenT0 = p_s_c_given_T0.sum(axis=0)
p_c_givenT1 = p_s_c_given_T1.sum(axis=0)
In [17]:
np.outer(p_s_givenT0, p_c_givenT0) == p_s_c_given_T0
##hmmm
Out[17]:
array([[False,  True],
       [False,  True]], dtype=bool)
In [18]:
p_s_c_given_T0
Out[18]:
array([[ 0.72,  0.08],
       [ 0.18,  0.02]])

Find $P_T(t)$

Using total probability to marginalize for t=0:

$\qquad P(s=0, c=0) = P(s=0,\ c=0\ |\ t=0)\cdot P(t=0) + P(s=0,\ c=0\ |\ t=1)\cdot P(t=1)$

can replace $P(t=1)$ with $1-P(t=0)$

$P(s=0,\ c=0)$

In [19]:
print('%.2f'%P_S_C[0,0])

0.40

$P(s=0,\ c=0 \ |\ t=0)$:

In [20]:
## given t=0, prob s=0,c=0
print('%.2f' % p_s_c_given_T0[0,0])

0.72

right now we have:

$\qquad0.40 = 0.72\cdot P(t=0) + P(s=0,\ c=0\ |\ t=1)\cdot(1-P(t=0))$

finding: $P(s=0\ ,c=0\ |\ t=1)$:

In [21]:
print('%.2f'% p_s_c_given_T1[0,0])

0.08

Now solving:

$\qquad 0.40=0.72\cdot P(t=0) + 0.08 - 0.08\cdot P(t=0)$

In [22]:
##Solve
P_T = {}
P_T[0] = (0.40-0.08)/(0.72-0.08)
P_T[1] = 1 - P_T[0]
P_T
Out[22]:
{0: 0.5, 1: 0.5}

Expectation

In [23]:
## simulating 1000 die rolls
import random
In [24]:
rolls = {i:0 for i in range(1,7)}
random.seed(1)
for _ in range(1000):
    roll = random.randint(1,6)
    rolls[roll] += 1
rolls
Out[24]:
{1: 160, 2: 166, 3: 165, 4: 163, 5: 183, 6: 163}
In [25]:
expected_val = np.array([k*v for k,v in rolls.items()]).sum() / 1000
expected_val
Out[25]:
3.532

Variance using expectations:
$\qquad var(X)=\mathbb{E} [X^2]- (\mathbb{E} [X])^2$

In [26]:
L1 = {-1: 999999/1000000, 999: 1/1000000}
L2 = {-1: 999999/1000000, 999999: 1/1000000}
L3 = {-1: 9/10, 9: 1/10}
In [27]:
## L1
import math
e_x2 = np.array([k**2*v for k,v in L1.items()]).sum()
e_x_2 = np.array([k*v for k,v in L1.items()]).sum()**2
sd_l1 = math.sqrt(e_x2-e_x_2)
e_x2 - e_x_2
Out[27]:
0.99999899999999975
In [28]:
## L2
e_x2 = np.array([k**2*v for k,v in L2.items()]).sum()
e_x_2 = np.array([k*v for k,v in L2.items()]).sum()**2
sd_l2 = math.sqrt(e_x2-e_x_2)
e_x2 - e_x_2
Out[28]:
999999.0
In [29]:
## L3
e_x2 = np.array([k**2*v for k,v in L3.items()]).sum()
e_x_2 = np.array([k*v for k,v in L3.items()]).sum()**2
sd_l3 = math.sqrt(e_x2-e_x_2)
e_x2 - e_x_2
Out[29]:
9.0
In [30]:
print('Standard Deivations:\nL1: %f\nL2: %f\nL3: %f\n' % (sd_l1,sd_l2,sd_l3))

Standard Deivations:
L1: 0.999999
L2: 999.999500
L3: 3.000000